Comparing Holistic and Analytical Rating Methods of Eliciting Preferences in Naming an Online Program Using Ranks as a Concurrent Validity Criterion

Current and prospective students (n =133) were surveyed about their preferences for a name for a new online series of courses to be launched by a university. Preferences for each of five names were solicited by means of analytical ratings, holistic ratings, and rankings. All three techniques were employed to assure that the most appropriate name for the program was selected, but this also afforded us the opportunity to study several theoretical issues: (a) Do the different methods lead to discrepant decisions at the aggregate level? (b) Is the holistic rating or the analytical rating approach more closely related to the rankings? (c) To what extent is lack of agreement between ratings and rankings due to lack of differentiation in ratings? The authors find that at the aggregate level all three methods suggest the same name for the program; the holistic rating is slightly more highly correlated with the ranking; and the lack of differentiation in ratings is one reason producing inconsistencies between ratings and rankings. DOI: 10.4018/978-1-4666-4014-6.ch017